1. Motivation

We were highly motivated by the extraordinare possiblity of working with the covid pandemic. It found it especially interesting because it was a very present issue that affected us all.

1.1 Datasets

1.2 Why did you choose this/these particular dataset(s)?

Amount of the corona related datasets can be overwhelming. The one from ECDC was chosen, because it was the most neutral one. Although, the famous dataset from Hopkins has almost the same data, and even complementary information such as specific geographic coordinates of cases, the level of the detail in the dataset seemed biased towards the USA. Likewise, the DOLT dataset was heavily biased towards Asia.

The ECDC corona data set provided a fine basis for examining the spread of the virus - measured by both: the number of registered cases and deaths. Further, the set was also coded with proper country codes.

The general country dataset was used mainly for two reasons. Firstly, out of the need to translate country codes into country names, but also to have a way to translate between different country coding styles. The chosen dataset was highly relevant in the early state, since at that point currency exchange rates were thought to be a central theme, and this data set also made it easily possible to relate currency codes to country codes.

Vector maps were necessary for map making. However, there are a lot of such datasets of varying quality. While many of such are based on the very same vector data, the data accompanying the vector features were often of questionable quality. Essentially it was preferable to trust other, more original sources and use only the provided vector data.

Although stories in the media referenced data regarding subjects such as quickly rising unemployment, this data was not easily accessible. In fact, it seemed not possible to gather such data for an up-to-date analysis of the global situation. This is probably due to the fact, that datasets about the economical state of each country, tend to be published as part of a periodical retrospective analysis. Therefore, other economic data, describing the day-to-day development in parallel to the spread of corona, had to be found.

Initially, currency exchange rates were chosen as a highly relevant data source. However, it turned out that such exchange rate datasets are of course relative to a chosen currency - typically USD. This would describe the global development relative to the development of the country, of the chosen currency, which could be problematic. Further on, currency exchange rates of course turned out to be a bad measurement for comparing the European countries due the to the usage of Euro across many of these countries. Even the Danish krona is functionally locked to the Euro. In the end, currency rates were abandoned in favor of government bonds.

However, government bonds also turned out to be an issue, since not all countries have short-lived government bonds. Among the selected focus countries some did only have 10-year bonds. Such a timeframe would definitely reduce the impact of the very present situation with the corona pandemic. Bonds were therefore also deselected for inclusion on the final production.

In the end, national stock indexes were chosen as an indicator for the present development in the countries. Using such indexes is not ideal since they only include a narrow selection of the most popular stocks. Also, the companies included are often actors on the global market, and the indexes have an unintended international bias. Further on, they do not contain or describe the smaller businesses nor privately owned. However, the indexes describe the investors willingness to invest in the most valued businesses of each country. In the end, for a smaller analysis like this, such bias is considered quite acceptable when trying to compare how the corona pandemic affect the individual countries.

The flight tracking, oil and natural gas datasets were chosen to complement the stock indexes, as other up-to-date sources describing the global effects of the pandemic. These were chosen simply because of how they showed an unprecedented, abrupt, global change.

Human Development Index is a statistic index, which aggregates information about: life expectancy, years of schooling and Gross national income (per capita) of the countries and expresses them in a value between 0 and 1. According to this metric as of 2018, most developed country in the world is Norway and least developed one is Niger. We wanted to investigate how such a socio-economic index correlate to the Covid outbreak.

Finally, none of the Covid cases would be recorded, if not for the test performed on patients. The tests allow the governments to track the outbreak and plan their next countermeasures. We considered, quite relevant to take a closer look at how many tests are done by each country, and if the amount of testing can lead to smaller amount of deaths.

1.3 What was your goal for the end user's experience?

We chose to do a magazine article type of production and intended to produce also some actual analyses. However, in the end we had not enough time to also produce such, and the production will therefore somewhat lacky on that form. Nonetheless, we believe that the visualizations would be a great support for the intended text of an article.

The goal for the reader should be to get a comprehensive overview about global Covid pandemic situation. By investigating the crisis from different angles, we are able to understand current affairs better and more in-depth. The user should be able to interact with visualizations provided, explore the topic in detail and perhaps draw some new conclusions.

2 Basic stats. - Let's understand the dataset better

2.1 Data cleaning and Preprocessing

The actual data import, cleaning and preprocessing, is done via the homemade python modules (own.data.*). Generally, such modules load csv or json data files into Pandas dataframes or series and store them in variables in the modules. Besides csv files, json files are also imported. These have the capacity to have nested lists and maps, which require more manual processing than two-dimensional csv files. Such is also done in the homemade modules.

Where relevant, data types for categorical columns are set already on import. Date columns are cast into datatime, and when dates are separated into date, month and year, these are merged into one column. Also, values such as NaN and booleans are cleaned into corresponding python data structures. Finally, column naming is typically fixed to an internal string.

For easy data retrieval, the modules also contain convenient methods that provide easy access to the data in various forms. This could be in cumulated form or merged with other data sets.

Generally, three-letter country codes are used across all data set whenever possible. The codes used was expected to simply follow the ISO 3166-1 alpha-3 standard, but abnormalities appeared since some countries are not recognized by all organizations. Kosovo (KOR) is for instance recognized by EU, but not internationally by UN. Also, there are issues with sovereignty, polluting some country coding.

2.2 Presentation of the datasets

In the folloowing we present the different structure and content of the different datasets.

2.2.1 ECDC's global corona dataset

The actual data import, cleaning and preprocessing, is done via the the homemade own.data.ecdc module as previously described.

The ECDC data set consists of one csv file, that will have to be imported into a pandas dataframe. The raw structure are af follows:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 15906 entries, 0 to 15905
Data columns (total 11 columns):
 #   Column                   Non-Null Count  Dtype  
---  ------                   --------------  -----  
 0   dateRep                  15906 non-null  object 
 1   day                      15906 non-null  int64  
 2   month                    15906 non-null  int64  
 3   year                     15906 non-null  int64  
 4   cases                    15906 non-null  int64  
 5   deaths                   15906 non-null  int64  
 6   countriesAndTerritories  15906 non-null  object 
 7   geoId                    15850 non-null  object 
 8   countryterritoryCode     15710 non-null  object 
 9   popData2018              15725 non-null  float64
 10  continentExp             15906 non-null  object 
dtypes: float64(1), int64(5), object(5)
memory usage: 1.3+ MB
dateRep day month year cases deaths countriesAndTerritories geoId countryterritoryCode popData2018 continentExp
0 09/05/2020 9 5 2020 215 3 Afghanistan AF AFG 37172386.0 Asia
1 08/05/2020 8 5 2020 171 2 Afghanistan AF AFG 37172386.0 Asia
2 07/05/2020 7 5 2020 168 9 Afghanistan AF AFG 37172386.0 Asia

A few countries have such as Anguilla and Nambia has missing country codes. For now, these are simply skipped, but could be preserved by manual data cleaning.

The date and country columns are then set a dual multi-index.

The population size of each country is extracted and preserved in a separate variable.

Finally, only relevant columns a preserved

<class 'pandas.core.frame.DataFrame'>
MultiIndex: 15654 entries, (Timestamp('2020-05-09 00:00:00'), 'AFG') to (Timestamp('2020-03-21 00:00:00'), 'ZWE')
Data columns (total 5 columns):
 #   Column     Non-Null Count  Dtype         
---  ------     --------------  -----         
 0   reported   15654 non-null  datetime64[ns]
 1   cases      15654 non-null  int64         
 2   deaths     15654 non-null  int64         
 3   name       15654 non-null  category      
 4   continent  15654 non-null  category      
dtypes: category(2), datetime64[ns](1), int64(2)
memory usage: 498.9 KB
reported cases deaths name continent
date country
2020-05-09 AFG 2020-09-05 215 3 Afghanistan Asia
2020-05-08 AFG 2020-08-05 171 2 Afghanistan Asia
2020-05-07 AFG 2020-07-05 168 9 Afghanistan Asia

Preprocessing

The own.data.ecdc module also contains a range of helper emthods that provide easy access to various combinations of the data. Some of these methods return the data unstacked()'ed, and some add cumulative values.

cases deaths
date country
2020-05-09 AFG 215 3
2020-05-08 AFG 171 2
2020-05-07 AFG 168 9
2020-05-06 AFG 330 5
2020-05-05 AFG 190 5
cases_total deaths_total
date country
2020-05-09 ZAF 8895.0 178.0
ZMB 167.0 4.0
ZWE 35.0 4.0
cases_total_per_capita deaths_total_per_capita
date country
2020-05-09 ZAF 0.000154 3.080671e-06
ZMB 0.000010 2.305233e-07
ZWE 0.000002 2.770271e-07
cases deaths cases_total deaths_total cases_total_per_capita deaths_total_per_capita
date country
2020-05-09 AFG 215 3 3778.0 109.0 0.000102 0.000003
2020-05-08 AFG 171 2 3563.0 106.0 0.000096 0.000003
2020-05-07 AFG 168 9 3392.0 104.0 0.000091 0.000003
date
country
AFG 2020-03-24
ALB 2020-03-12
DZA 2020-03-13
population
country
AFG 37172386.0
ALB 2866376.0
DZA 42228429.0

The number of daily deaths per country vary a lot.

However, so does the size of the populations

2.2.2 Vector maps

Geospacial data such as country borders, are imported via geojson files, that contain features (rows) with both a geospatial column and possibly also other data. These are imported with GeoPandas, a geospacial extension of Pandas, which makes the data mergable with other data frames. The geojson files used, also contain some relevant ordinal columns, and these are properly ordered. Also, the file includes some convenience columns for map coloring, but thse are not used. Most of the columns from the geojson fiels were not used, but left as is. In the end the prime relevance were the geospacial columns.

The actual import and porcessing is done via the homemade own.data.countries module.

<class 'geopandas.geodataframe.GeoDataFrame'>
Index: 175 entries, AGO to PNG
Data columns (total 15 columns):
 #   Column          Non-Null Count  Dtype   
---  ------          --------------  -----   
 0   iso3166_a3      172 non-null    object  
 1   iso3166_a2      172 non-null    object  
 2   iso3166_n3      172 non-null    object  
 3   name_en         175 non-null    object  
 4   name_long_en    175 non-null    object  
 5   name_formal_en  173 non-null    object  
 6   label_rank      175 non-null    int64   
 7   tiny            9 non-null      float64 
 8   mapcolor_7      175 non-null    int64   
 9   mapcolor_8      175 non-null    int64   
 10  mapcolor_9      175 non-null    int64   
 11  mapcolor_13     175 non-null    int64   
 12  economy         175 non-null    category
 13  income_group    175 non-null    category
 14  geometry        175 non-null    geometry
dtypes: category(2), float64(1), geometry(1), int64(5), object(6)
memory usage: 20.0+ KB
iso3166_a3 iso3166_a2 iso3166_n3 name_en name_long_en name_formal_en label_rank tiny mapcolor_7 mapcolor_8 mapcolor_9 mapcolor_13 economy income_group geometry
id
AGO AGO AO 024 Angola Angola People's Republic of Angola 3 NaN 3 2 6 1 7. Least developed region 3. Upper middle income MULTIPOLYGON (((16.32653 -5.87747, 16.57318 -6...
BDI BDI BI 108 Burundi Burundi Republic of Burundi 6 NaN 2 2 5 8 7. Least developed region 5. Low income POLYGON ((29.34000 -4.49998, 29.27638 -3.29391...
CIV CIV CI 384 Côte d'Ivoire Côte d'Ivoire Republic of Ivory Coast 3 NaN 4 6 3 3 6. Developing region 4. Lower middle income POLYGON ((-2.85613 4.99448, -3.31108 4.98430, ...

2.2.3 Country data

Country data aggregates data from: governmental agencies, UN and some independent projects, about countries in the world. It helps us by providing mapping between country names and country codes. As the common names tend to vary in different sources, one parameter which usually remains the same is the country code (either two-letter or three letter). Furthermore, it includes information about continents, regions and sub-regions, which allow to group countries together, as well as development classification (established by United Nations) and country independency status (established by CIA).

Country data was placed in the module own.data.countries

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 250 entries, 0 to 249
Data columns (total 18 columns):
 #   Column                            Non-Null Count  Dtype   
---  ------                            --------------  -----   
 0   official_name_en                  249 non-null    object  
 1   ISO3166-1-Alpha-2                 248 non-null    object  
 2   ISO3166-1-Alpha-3                 249 non-null    object  
 3   ISO3166-1-numeric                 249 non-null    float64 
 4   ISO4217-currency_alphabetic_code  244 non-null    object  
 5   ISO4217-currency_name             247 non-null    object  
 6   ISO4217-currency_numeric_code     244 non-null    object  
 7   CLDR display name                 249 non-null    object  
 8   Capital                           243 non-null    object  
 9   Continent                         208 non-null    category
 10  Developed / Developing Countries  248 non-null    object  
 11  ITU                               233 non-null    object  
 12  Least Developed Countries (LDC)   250 non-null    bool    
 13  Region Code                       248 non-null    category
 14  Region Name                       248 non-null    category
 15  Sub-region Code                   248 non-null    category
 16  Sub-region Name                   248 non-null    category
 17  is_independent                    249 non-null    category
dtypes: bool(1), category(6), float64(1), object(10)
memory usage: 26.2+ KB
official_name_en ISO3166-1-Alpha-2 ISO3166-1-Alpha-3 ISO3166-1-numeric ISO4217-currency_alphabetic_code ISO4217-currency_name ISO4217-currency_numeric_code CLDR display name Capital Continent Developed / Developing Countries ITU Least Developed Countries (LDC) Region Code Region Name Sub-region Code Sub-region Name is_independent
0 NaN TW TWN 158.0 NaN NaN NaN Taiwan Taipei AS NaN NaN False NaN NaN NaN NaN Yes
1 Afghanistan AF AFG 4.0 AFN Afghani 971 Afghanistan Kabul AS Developing AFG True 142 Asia 34 Southern Asia Yes
2 Albania AL ALB 8.0 ALL Lek 008 Albania Tirana EU Developed ALB False 150 Europe 39 Southern Europe Yes

Preprocessing

As we mentioned in the description above, the dataset is very comprehansive. Thus we don't always will need all the columns. Below we show, which information can be extracted from the main dataframe with the module.

offical_name common_name
country
TWN NaN Taiwan
AFG Afghanistan Afghanistan
ALB Albania Albania
continent region_name subregion_name region_code subregion_code
country
TWN AS NaN NaN NaN NaN
AFG AS Asia Southern Asia 142 34
ALB EU Europe Southern Europe 150 39
DZA AF Africa Northern Africa 2 15
ASM OC Oceania Polynesia 9 61
... ... ... ... ... ...
ESH AF Africa Northern Africa 2 15
YEM AS Asia Western Asia 142 145
ZMB AF Africa Sub-Saharan Africa 2 202
ZWE AF Africa Sub-Saharan Africa 2 202
ALA EU Europe Northern Europe 150 154

250 rows × 5 columns

The dataset also containes details about the countries development status, which is made available separately

development_type least_developed
country
TWN NaN False
AFG Developing True
ALB Developed False

2.2.4 Oil prices

Oil data we found, comes from two sources and shows prices in USD per barrel). The first one (BRENT) refers to European oil prices in years 1987-2020, while the second datset (WTI CRUDE) to American ones in years 1986-2020. Oil data is stored in the module: own.data.oil

Price
Date
2020-04-30 18.11
2020-05-01 18.49
2020-05-04 20.40 
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 8367 entries, 1987-05-20 to 2020-05-04
Data columns (total 1 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   Price   8367 non-null   float64
dtypes: float64(1)
memory usage: 450.7 KB

Preprocessing

Because we wanted to keep our analysis global and also because the difference between those regions is not significant we calculated the average oil price and use it further analysis. Oil prices dataset is stored in the module: own.data.oil

Breant Twi Average
Date
2020-04-30 18.11 19.23 18.670
2020-05-01 18.49 19.72 19.105
2020-05-04 20.40 20.47 20.435 
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 8784 entries, 1986-01-02 to 2020-05-04
Data columns (total 3 columns):
 #   Column   Non-Null Count  Dtype  
---  ------   --------------  -----  
 0   Breant   8367 non-null   float64
 1   Twi      8654 non-null   float64
 2   Average  8784 non-null   float64
dtypes: float64(3)
memory usage: 274.5 KB

2.2.5 Natural Gas prices

Gas data is provided by U.S. Energy Information Administration EIA and provides an overview of gas prices (in USD) in the United States in years 1997-2020. Data is stored in the module: own.data.gas

Price
Date
2020-05-01 1.69
2020-05-04 1.78
2020-05-05 1.93 
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 5871 entries, 1997-01-07 to 2020-05-05
Data columns (total 1 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   Price   5870 non-null   float64
dtypes: float64(1)
memory usage: 91.7 KB

2.2.6 Flights

This dataset is combined out of two subsets. First one describes total amount of flights. This basically includes all the possible flying objects flightradar was able to track and store in database. While, the second is narrowed down to commercial passenger, cargo, charter and some business flights. The actual raw data consists of more than one file, as we gathered the data over few days (flight radar tracks down the flights only 90 days back). Therefore, the presented data is actually a couple of csv files accumulated into two raw datasets. Two datasets are rather similar and contain total number of flights as well as 7-day average. Flights data is stored in the module: own.data.flights

Total flights

7day_average count
date
2020-01-22 180920 184248
2020-01-23 181318 189810
2020-01-24 181517 189551

Commercial flights

7day_average count
date
2020-04-21 27984 29436
2020-01-22 111905 113739
2020-04-20 28085 26663

Preprocessing

The only processing done at this stage is two combine the dataframes into one, which has all the gathered data. As one can observe below we are missing commercial flights data. This was due to the inclusion of a an older data file, on total flights only.

commercial_7day_average commercial total_7day_average total
date
2020-01-14 NaN NaN 172803 173162
2020-01-15 NaN NaN 173944 187297
2020-01-16 NaN NaN 174154 187023
2020-01-17 NaN NaN 174764 188157
2020-01-18 NaN NaN 175720 161254
... ... ... ... ...
2020-05-04 29658.0 29773.0 76318 77484
2020-05-05 30094.0 33930.0 76833 84315
2020-05-06 30359.0 34128.0 78821 90071
2020-05-07 30559.0 34907.0 81337 96122
2020-05-08 30980.0 34514.0 82373 88776

120 rows × 4 columns

2.2.7 Stock index'es

Stock data are loaded form multiple json files - one per country. While the whole original time series are kept, only the closing values are processed and used. These are processed into one collective pandas dataframe.

The actual import and processing is done via the homemade own.data.stocks2 module.

{'CHN': 'Shanghai Stock Exchange Composite Index',
 'DEU': 'DAX German Stock Index',
 'DNK': 'OMX COPENHAGEN',
 'ESP': 'IBEX 35',
 'FRA': 'Cotation Assistée en Continu 40',
 'GBR': 'FTSE 100',
 'ITA': 'FTSE Milano Italia Borsa',
 'SWE': 'OMXS 30 Stockholm',
 'USA': "Standard & Poor's 500"}
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 1256 entries, 2015-05-18 to 2020-05-11
Data columns (total 8 columns):
 #   Column         Non-Null Count  Dtype  
---  ------         --------------  -----  
 0   open           1256 non-null   float64
 1   high           1256 non-null   float64
 2   low            1256 non-null   float64
 3   close          1256 non-null   float64
 4   x              1256 non-null   int64  
 5   y              1256 non-null   float64
 6   percentChange  1255 non-null   float64
 7   change         1255 non-null   float64
dtypes: float64(7), int64(1)
memory usage: 88.3 KB
open high low close x y percentChange change
date
2020-05-06 1164.76 1174.76 1156.60 1158.72 1588723200000 1158.72 0.543186 6.26
2020-05-07 1162.73 1178.92 1161.99 1170.32 1588809600000 1170.32 1.001105 11.60
2020-05-11 1181.32 1187.63 1172.61 1187.63 1589155200000 1187.63 1.479083 17.31
CHN DEU DNK ESP FRA GBR ITA SWE USA
date
2020-05-05 NaN 10729.46 1152.46 6750.85010 4475.75977 5849.42 17387.38 1538.178 2868.44
2020-05-06 2878.1401 10606.20 1158.72 6675.83008 4431.99023 5853.76 17159.31 1521.400 2848.42
2020-05-07 2871.5232 10759.27 1170.32 6734.81006 4497.87988 5935.98 17245.04 1537.735 2881.19
2020-05-08 NaN NaN NaN NaN NaN NaN 17439.30 1563.188 2929.80
2020-05-11 2894.8020 10824.99 1187.63 6672.14014 4488.64990 5939.73 17381.36 1556.488 2940.00

2.2.8 Testing

Testing data from OWID contains details about not only abut testing, but also cases and deaths, as well as their normalization. Timespan of the dataset spreads between the and on December 2019 and end of April 2020. The module responsible for loading this data: own.data.owid

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 12874 entries, 0 to 12873
Data columns (total 16 columns):
 #   Column                    Non-Null Count  Dtype  
---  ------                    --------------  -----  
 0   iso_code                  12874 non-null  object 
 1   location                  12874 non-null  object 
 2   date                      12874 non-null  object 
 3   total_cases               12874 non-null  int64  
 4   new_cases                 12874 non-null  int64  
 5   total_deaths              12874 non-null  int64  
 6   new_deaths                12874 non-null  int64  
 7   total_cases_per_million   12665 non-null  float64
 8   new_cases_per_million     12665 non-null  float64
 9   total_deaths_per_million  12665 non-null  float64
 10  new_deaths_per_million    12665 non-null  float64
 11  total_tests               2966 non-null   float64
 12  new_tests                 2630 non-null   float64
 13  total_tests_per_thousand  2966 non-null   float64
 14  new_tests_per_thousand    2634 non-null   float64
 15  tests_units               2966 non-null   object 
dtypes: float64(8), int64(4), object(4)
memory usage: 1.6+ MB

Processing

We filter out information, which is not test related, as we take the data on COVID from ECDC dataset. After further, inspection we notice that 128 countries lack infromation about tests. Furthermore the units of the tests varies a lot between countries, as OWID classifies them into 10 different categories.

[Aruba, Afghanistan, Angola, Anguilla, Albania, ..., Vietnam, Yemen, South Africa, Zambia, Zimbabwe]
Length: 206
Categories (206, object): [Aruba, Afghanistan, Angola, Anguilla, ..., Yemen, South Africa, Zambia, Zimbabwe]
[Argentina, Australia, Austria, Belgium, Bangladesh, ..., Uganda, Uruguay, United States, Vietnam, South Africa]
Length: 78
Categories (78, object): [Argentina, Australia, Austria, Belgium, ..., Uruguay, United States, Vietnam, South Africa]

2.2.9 Human Development Index

HDI dataset provides values of HDI for 190 countries in years 1980-2017, together with country codes and names. Datset is loaded and processed by module: own.data.hdi

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 5001 entries, 0 to 5000
Data columns (total 4 columns):
 #   Column                              Non-Null Count  Dtype  
---  ------                              --------------  -----  
 0   Entity                              5001 non-null   object 
 1   Code                                4973 non-null   object 
 2   Year                                5001 non-null   int64  
 3    ((0-1; higher values are better))  5001 non-null   float64
dtypes: float64(1), int64(1), object(2)
memory usage: 156.4+ KB
Country_name Country_code Year HDI_value
0 Afghanistan AFG 1980 0.228
1 Afghanistan AFG 1985 0.273
2 Afghanistan AFG 2002 0.373
3 Afghanistan AFG 2003 0.383
4 Afghanistan AFG 2004 0.398
... ... ... ... ...
4996 Zimbabwe ZWE 2013 0.516
4997 Zimbabwe ZWE 2014 0.525
4998 Zimbabwe ZWE 2015 0.529
4999 Zimbabwe ZWE 2016 0.532
5000 Zimbabwe ZWE 2017 0.535

5001 rows × 4 columns

Preprocessing

In order to correlate the HDI data to other datsets we have merged it with: country data, tests and ecdc data. This allows us to easily corralate various data or group countries by regions, continents or economic factors. We also caluclate tests per capita.

<class 'pandas.core.frame.DataFrame'>
Int64Index: 74 entries, 0 to 74
Data columns (total 21 columns):
 #   Column                   Non-Null Count  Dtype         
---  ------                   --------------  -----         
 0   index                    74 non-null     object        
 1   Country_name             74 non-null     object        
 2   Year                     74 non-null     float64       
 3   HDI_value                74 non-null     float64       
 4   date                     74 non-null     datetime64[ns]
 5   cases_total              74 non-null     float64       
 6   deaths_total             74 non-null     float64       
 7   population               74 non-null     float64       
 8   tests_total              74 non-null     float64       
 9   tests_units              74 non-null     object        
 10  continent                67 non-null     category      
 11  region_name              74 non-null     category      
 12  subregion_name           74 non-null     category      
 13  common_name              74 non-null     object        
 14  development_type         74 non-null     object        
 15  tests_per_capita         74 non-null     float64       
 16  cases_total_per_capita   74 non-null     float64       
 17  deaths_total_per_capita  74 non-null     float64       
 18  iso3166_a2               74 non-null     object        
 19  economy                  74 non-null     category      
 20  income_group             74 non-null     category      
dtypes: category(5), datetime64[ns](1), float64(9), object(6)
memory usage: 11.9+ KB

2.2.10 Original (borrowed) stock index'es

During initial data analysis, another group's stock data were used as inspiration. This set were made available via Slack. The excel file required different processing, since the data was not provided with dates per row, but rather by column. The actual import and processing is done via the homemade own.data.stocks module.

In the end, since data was limited to April 15th 2020, it was only used for inspiration and picking out relevant stock indexes per country.

Index(['SHCOMP', 'ATX', 'BEL20', 'OMXC25', 'HEX25', 'CAC', 'FTSEMIB', 'DAX',
       'ASE', 'ICEX', 'ISEQ', 'AEX', 'WIG', 'PSI20', 'IBEX', 'OMX', 'SM',
       'UKX', 'SPTSX', 'SPX', 'AS51', 'NZX20', 'HS', 'NKY', 'KOSP', 'ST',
       'TWSE', 'IBOV', 'TA-35', 'SENSEX', 'MEXBOL', 'JALSH', 'MOEX10'],
      dtype='object', name='index')

2.2.11 Governmental 10Y Bonds

AUS AUT BEL BRA CAN CHE CHN DEU DNK ESP ... POL PRT RUS SGP SWE TWN USA ZAF HKG NOR
date
2020-05-06 0.92 -0.072 0.093 7.67 0.614 -0.506 2.555 -0.502 -0.284 0.921 ... 1.564 0.990 6.10 0.932 -0.091 0.495 0.713 9.780 0.661 0.568
2020-05-07 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... 1.545 0.954 6.12 0.932 -0.108 0.500 0.631 9.445 0.698 0.449
2020-05-08 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN ... 1.536 0.933 5.95 0.889 -0.085 0.490 0.681 9.290 0.610 0.423

3 rows × 34 columns

2.2.12 Currency exchange rates - ECB (European Central Bank)

Timestamp('1999-01-04 00:00:00')
USD JPY BGN CYP CZK DKK EEK GBP HUF LTL ... ILS INR KRW MXN MYR NZD PHP SGD THB ZAR
Date
2020-04-24 1.0800 116.22 1.9558 NaN 27.310 7.4577 NaN 0.87498 355.19 NaN ... 3.7985 82.3660 1332.59 26.8331 4.7093 1.7991 54.891 1.5396 35.030 20.5019
2020-04-23 1.0772 115.75 1.9558 NaN 27.551 7.4577 NaN 0.87200 357.51 NaN ... 3.8207 81.9400 1326.80 26.3418 4.6966 1.7947 54.571 1.5331 34.815 20.5432
2020-04-22 1.0867 117.01 1.9558 NaN 27.534 7.4589 NaN 0.87920 355.11 NaN ... 3.8500 83.1685 1339.66 26.3895 4.7437 1.8200 55.096 1.5508 35.176 20.4413

3 rows × 41 columns

3. Data Analysis

3.1 ECDC - The corona spread

One thing is the number of daily deaths, another is how these cumulate over time. During the previous data exploration, it was obvious that USA generally had more daily deaths and therefore would have also more total deaths. The other countries were a blurrier story. It turns out that a group of larger western countries dominates the charts.

However, as the population size varies, so would the number of deaths. When calculating the total number of deaths per capita, there is quite a different distribution. Two very small states, San Marino and Andorra now appear at the top three, but the order of the other countries has shifted a great deal. Most noticeably, USA now by no means the leading country. Surprisingly, Belgium is taking the lead (if the two micro-states are ignored).

These bar plots do, however, not provide proper insight into the development over time. Plotting the timeseries of cumulated number of deaths of each of the 30 worst affected countries, illustrates how differently the pandemic has spread across these countries. But yet again the per-capita-version show that other side of the truth.

Limiting the number of countries to a selection of 10 focus countries further highlights this difference.

For the record: the selection mainly includes a mix of mainly European countries that. When measured by total number of deaths could roughly be split into three groups, but when measured per capita, they spread out nicely and with a different order. USA and Belgium were specifically selected as the most affected countries. China for being the patient-0 country. Denmark and Sweden were selected due to their overall similarity, yet noticeable differences in countermeasures against Corona.

3.2 Flights, Oil and Natural Gas

These datasets were already plotted in the previous section, but with a much longer timeseries that necessary. When focusing on only the recent months, it is clear that the oil prices were apparently affected, but that the gas prices have actually been on a steady level during the corona crises.

3.3 Flights

These datasets were already plotted in the previous section, and the plot clearly showed how the number of registered flights fell as the epidemic turned into a pandemic. There should be no further analyses necessary.

3.4 Stocks

As we have seen in the previous chapter, it might be hard to compare absolute values for countries. Similar behavior occurs in stocks. The situation is even more complicated, as the values are determined on the country level, of the national stock exchange. Therefore, we decided to normalize each index over the value it had in the beginning of 2020.

By plotting the normalized values, we are able to understand how stock exchanges were impacted by COVID and compare, which countries were affected the most. It is clear that China's index was impacted first, as the outbreak there began earlier then in Western countries. China experienced a second decline around the time the Western countries experienced it too. However, it's the smallest of all the countries shown in the graph. We can observe how stocks start to drop around the end of February, when the first deaths in Italy and Spain were recorded. The decrease in value continues for around two weeks, until the mid-March, when many European countries started to enforce lockdowns. What makes this figure interesting, is the notable increase of value in the end of March, when the restrictions were becoming more and more severe. This tendency continues, with varied gain, till the beginning of May.

If we look at the latest days shown in the plot, we can easily name countries indexes, which were least and most impacted by the outbreak. Three most impacted ones are: Spanish, Italian and French indexes, while three least impacted are: Danish, Chinese and American ones.

3.5 Human development index (HDI), compared to COVID data

In order to combine data about cases, deaths and tests, with HDI we had to make a choice exclude some countries. In particular those, which didn't have any data on tests. Furthermore, we were interested in the most recent data, but the actual time span of the dataset is between 13th and 23rd of April. This due to the fact, the for each of the countries, we recorded only the latest date on which all three values (cases, deaths and tests) were available. In summary in this analysis we consider 74 countries all over the world.

3.5.1 Total vs per capita

As we have seen previously with cases and deaths, total numbers might be not a fair metric when comparing country related data. Therefore, in further analysis we shall only use values per capita.

Let's take a look at tests alone. Countries, which perform a lot of tests per capita are either small, located in Europe or both. As one might expect, underdeveloped and poor countries perform significantly less tests. This leads us to conclusion that, it's hard to estimate the impact of COVID in those parts of the world.

Finally, if we look at the focus countries, we observe that Italy performs the most, while the UK the least tests per capita.

3.5. Tests vs HDI

As the final remark it is important to understand, why some countries perform less tests. Human development index might give us an answer. The more the developed the country, the more tests it can perform, or better - can afford. According to this source the price range of a test in the USA is between 35-50 USD. So, as long as those tests will not become affordable or accessible for the developing countries, we won't be able to understand how badly is the COVID crisis impacting them.

4 Genre

4.1 Choice of genre of data story

Such complex topic like COVID-19 outbreak, could be explained in all possible genres of visualizations. We decided to choose the old-fashioned magazine style. It allows to include meaningful visualizations, while leaving some room for explanation written in text.

Either way, we believe that the production serves the purpose of providing the reader relevant information about the spread corona pandemic, how it impacts a small handfull of economical aspects on both the global and national level, and even how differently the situation is experienced in developing and developed contries. This last metric is specially meaningful for the test data, as the testing allows countries to monitor and track the health crisis.

4.2 Visual Narrative tools

Visual Structuring

  • Consistent Visual Platform
    • Most of the plots are done with the same library, with similiar functionalities, when it comes to moving and navigating through the plot. This allows the reader to get used to it and use it through out the website.

Highlighting

  • Feature Distinction
    • Legends describe distinct visual apperence of the individual features on the plots.
    • On plots that show focus countries, the same colors are used on all plots.
  • Zooming
    • In most plots it is possible to perform some kind of zooming or out. In many plots additional data is provided 'outside' of the initial window, and this is accessible by either zooming out or by panning vertically. Maps natually offer both zooming and panning in a customary fashion.
    • On each plot is is possible to 'zoom' to at full-screen version of the plot.

Transition Guidance

  • None

4.3 Narrative Structure

Ordering

  • User Directed Path
    • Our goal was to explain COVID outbreak through different perspectives. We try to preserve an order, which allows for the story to grow and evolve, from easy to understand maps to details on stock prices and COVID data.

Interactivity

  • Hover Highlighting / Details

    • In addition to visual zooming, it is also possible to zoom in on data through pop-ups. These show additional or more detailed data.

  • Filtering / Selection / Search

    • Via the legends on most plots, it is possible to mute data series individually. Hereby the reader can customize the plot to show only data of interest.

  • Navigation

    • We provide an easily usable navigation bar at the top of the page. Since we have produces just a one-page webside, we simply use the top-level navigation for in-document natigation. Had this been a real magazine, navigation would of cause had been realized in multiple layers.
    • Under each plot, there is a link to a full-screen version of the plot.

  • Buttons

    • One of visualization allows the user to pick data for x and y ranges. This allows us to inlclude multiple plot in one window and relate various subsets of data to each other.

  • Stimulating Default Views

    • Plots are per default zoomed in on the most recent parts of the data. This allows us to point attention of the reader to given time window, relevant to our main topic.

Messaging

  • Captions / Headlines

    • Since our intention was to produce a magazine article, the usage of headlines is a natural tool to make the thematic changes between sections easily identifiable.

  • Introductory Text

    • Each of the visualization is introduced and breifly explained, so that the reader understands what data is diplayed.

  • Summary / Synthesis

    • We have tried to provide a written overview of each plot, and extract the most important points and findings. However we leave some room for exploration to the user.

5. Visualizations: Choises and why they are right for the story

Rather than just referring to the final visualizations, we choose to also generate and display the very thing. The plots are generated by a handfull of notebooks, that is prparet for both saving plots to html files, showing the plots in the notebook, or saving them in a special variable for usage elsewhere. Here the later is of cause the case. A special python module (own.funny), which is also used by all these notebooks, changes the behavior of the notebooks.

Loading BokehJS ...
Loading BokehJS ...
Loading BokehJS ...
Loading BokehJS ...

5.1 Static Choropleth

We have chosen to start out with a choropleth map that show the current spread of the corona virus with a colormap that really highlights the differences between the countries. The counters are affected to a very different extend and using a traffic-light colormap that mirrors that the user would normally see on google maps' the traffic overlay, we seek to assist the reader into quickly understand the situation.

A choropleth is an often-good way to visualize a geographical difference, when these can be split into non-overlapping polygons. By starting out with such a visualization, we naturally also seek to catch the readers interest.

The choropleth plugin provides the basic mapping functionalities such a background image and zooming and panning capabilities in a way that the readier is accustomed to. On top of this we add a layer of polygons that match the countries, and coloring these according to the colormap. Also, we add tooltip hover boxed that contains various details about the country the curser is currently above. Thereby we prove a way for the reader to also zoom into the data and not only zoom in on the map.

5.2 Timed choropleth

However, a static choropleth does not give any insights into the temporal dimension. We therefore compliment the former choropleth with an animated version on which the reader can control time with a slider. We would have preferred adding the possibility of passing through time automatically, but in the end, we had to choose between different tools that each had their limitations, and we chose to lose automatic animation in favor of a tool that provided more appealing visualization with less effort.

For the times choropleth we neutrally use the very same colormap as the previous choropleth. However, we compliment the gradual change in color with a gradual increase in alpha value. Thereby we start out with a naked map, and the colored country polygons individual fades in as people begin to die in the corresponding country.

The plot only offers little functional interactivity: A hover tooltip shows the population size with higher precision than the value located on top of each bar.

This particular plot has show to be a bit troublesome when added like this, so the generated html file is also shown as a backup...

5.3 Population bar plot

We could easily have kept on using choropleth for also showing country population size, and by doing so, we could present the size of all countries. However, we have chosen a set of focus countries, and a bar plot is better suited for showing the actual size difference. With a choropleth the reader would have to interpret a color into quantity/amount, but with a bar plot the size difference is visualized in a way show the differences in a way that make it much easier to determine size differences - and also comparing them.

5.4 Line plots of death

Now, development of the virus in the ten focus countries are visualized together on two line plots. First, the total number of deaths per country is shown on one plot, and after that, another diagram shows deaths normalized to per capita. By displaying total deaths and deaths per capita on two different plots close to each other, it is possible to see observer the obvious difference in development.

By using distinct colors per country, the reader is assisted in locating the same country on the other plot - and on later plots.

Compared to the choropleths, the first plot with total deaths show the exact development per day per country on the y-axis, and not just a color which would only give an estimated value. Thereby it is possible to identify even small differences between two countries, which would be hard or even impossible on a choropleths.

By showing only a handful of focus countries instead of the whole lot, makes it possible to do a line plot like this in a meaningful way.

The legend allows for 'muting' the individual country. This would have allowed for inclusion of further countries and still have a meaningful uncluttered plot. However, the legend would grow linearly alongside the number of lines, and also the usage of distinct coloring per country would be lost.

Locating the y-axis on the right side, where the number of deaths grow, makes reading the latest values easier than if the z-axis were located on the left.

The plot contains further data that initially shown, since it has been zoomed into the most interesting period. It is therefore possible to pan to the past and zoom further in on the earlier stages.

And finally, a hover tooltip box of cause allows for exact reading of dates and deaths.

Altogether, the two plots successfully demonstrates the differences between development in total deaths and deaths per capita.

5.5 Tracked flights

By combining the global death count with the number of registered flights, it is possible to give the reader a feeling of just how closely the abrupt decline in airline traffic corresponds to the sudden increase in the global death count.

While the 7-day average is properly the most relevant, the inclusion of also daily counts on thinner and semi-transparent lines, the primary 7-day average lines is somewhat legitimized. And by pre-muting the daily lines, they are reduced to a noticeable shadow that does not call for attention, but still serve their purpose.

Since the y-values of the two datasets are quite different, a secondary axis is added to the right.

Altogether, the plot gives a good impression how the decrease in airline traffic caused a sudden rise in corona related deaths. just kidding, naturally...

5.6 Oil and Natural Gas prices

The plot with oil and natural gas prices basically follows the same pattern as the previous plot.

Contrary the flight traffic plot, this plot contains a long history of oil and gas prices. Plot is pre-zoomed into the most recent prices, since these are the relevant ones, but it is possible to pan back in time and zoom, should the reader be interested.

5.7 Stock values

The plot shows values of most important stock indexes for each of the focus countries. We used a line plot, which is a common choice for visualizing time-series. By showing the deaths, we are able to prove that the biggest companies are not impacted by increasing number of deaths worldwide. However, it might be hard to investigate the situation in different countries, because absolute values are not comparable to each other, as the indexes are listed on national stocks exchanges.

5.7b Stock values #2

We continue with the time-series visualizations with line plot, but this time we normalize over fixed data of the 2nd of January of 2020. This date was chosen as a reference point, because it was the very beginning of the COVID outbreak. It was also the time, when the markets weren't reacting to the crisis just yet. Here the lines become a little occluded, so the user is able to select the countries they are interested in by clicking on the legend.

5.8 HDI and test widget

Which choose scatter plot for this visualization as it provides a good of overview of the 74 countries, which published their data on COVID testing. Both axes are customizable, as the user can set different categories (cases, deaths and tests per capita) on Y axis. While, for the X axis user is able to choose between human development (HDI) and tests per capita. This allows to not only understand how cases and death depend on development level but also on tests performed by each country. With this visualization, we can i.e. see if the monitoring (tests) can lead to decrease in deaths. With the hover tool the reader is able learn the important details about the given country. Finally, if the plot becomes a little occluded, the user is able to choose the group of countries belonging to few of 7 development categories and hide the others.

6 Discussion

Choosing corona-related datasets as basis for our final project showed to be a risky choice. The outbreak is still developing and therefore the situation can change over a day or a week. We have tried, to find the best datasets, which would allow us to explore the impact of the epidemic in various areas of economy, but we also understand the spread of the disease better by looking at tests and HDI datasets. Searching for good data sets was quite time consuming, and in the end a many of the datasets identified were not used in the final website.

There are couple of aspects, we are glad we archived. With our article, the reader is able to get an impression of the extend covid crisis. In all sections - from the spread of the virus measured by the number of deaths, over the financial impact of the pandemic - the readers can compare situation in the different countries. And finally, and finally the reader can explore the data themselves thorough interactive a small dashboard describing how testing can be related to the Human Development Index.

However, our website by no means perfect. We initially chose to do a magazine style article with a good handful of visualizations, but in the end, we did not have time to do anything near the amount of article text as we had planned. We therefore ended up with a production that is kind of a mix between an article and annotated charts.

Though the intention was an article where the illustrations supported the text, and not the other way around, either way, we believe that the production servers the purpose of providing the reader relevant information about the spread corona pandemic, how the pandemic impacts a small handful of economic aspects on both the global and national level.

As for the visualizations, we have chosen to limit the number of countries to a handful of focus countries. We could have expanded the visualizations to include a few more countries or made the plots customizable with a complete list of all countries, so that the reader decides, which countries exactly shall be investigated. However, since our intention was a magazine article with a story to tell and not a dashboard, we chose to limit the visualizations to the what was actually needed for the story.

The spread of the pandemic can of cause also be related to population density of a given country, or their demographic composition. Such metrics were also considered, but since we had chosen an economical angle, they were considered out of our scope. Another, topic interesting to explore, would have been how restrictions applied by the countries were set according to the growth in cases and deaths. Furthermore, in our analysis we completely omitted the data on recoveries.

From the economical point of view, there a couple of indicators of how badly the national economies were impacted by the outbreak. The stocks we have found, might not be a good measure of countries economical situation, as they tend to operate worldwide, and their value might not rely solely on national market. The data on unemployment rate, retail sales or increase of national debt, could tell us more about how countries lessen the influence of the outbreak on the economy or how deeply they go into recession. Unfortunately, the only sources we found on those topics were either paid or not up to date.

A general issue with a lot of the datasets was that subjects such as for instance testing and countermeasures does not provide data that is easily comparable. Furthermore, the quality of the health care system in many countries are at best questionable, which could explain why some poorer countries have registered less deaths. However, the uneven spread might in fact be the reality, and would then instead be explained by for instance the richest countries travel pattern. In the end, based on our datasets, we are of cause not able to deduce any such conclusion.

7 Contributions

Overall, the project could be individualized as follows:

Heino

  • Corona related deaths (choropleth maps and line plots)
  • Oil prices
  • Gas prices
  • Flight tracking
  • Stocks
  • Overall website

Aleksander

  • Stocks
  • Human Development Index
  • Corona testing

Preprocessing has to a vast extend been structures collectively.